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1.0  INTRODUCTION 


1.1  BACKGROUND 

The  Air  Force  recognized  the  need  for  an  information  analysis  center  to 
serve  the  government,  industrial,  and  university  communities  as  a  focal  point 
for  software  development  and  experience  data  in  the  early  1970s.  In  1976  the 
Rome  Air  Development  Center  (RADC)  contracted  with  I  IT  Research  Institute 
(IITRI)  to  design  a  center  that  would  acquire,  analyze,  synthesize,  and 
disseminate  information  on  software  engineering  technology.  Subsequently, 
in  August  of  1978,  RADC  contracted  with  IITRI  to  develop  such  a  center,  which 
was  named  the  Data  &  Analysis  Center  for  Software  (DACS).  The  activities, 
accomplishments,  and  history  of  the  development  of  the  DACS  during  its  36- 
month  pilot  period  from  August  1978  through  August  1981  were  reported  in 
RADC-TR-81-385,  Establishment  of  the  Data  and  Analysis  Center  for  Software. 

The  DACS  was  designated  a  Department  of  Defense  (DOD)  Information 
Analysis  Center  (IAC)  in  January  1981  while  still  in  its  pilot  period.  At 
the  end  of  the  36-month  pilot  period,  IITRI  was  awarded  a  contract  to  operate 
the  DACS  for  an  additional  14-month  period.  The  primary  focus  of  this  effort 
was  to  provide  an  orderly  transition  from  a  pilot  information  center  to  a 
full  scale  IAC  and  from  a  center  completely  supported  by  government  funds  to 
an  IAC  whose  users  are  required  to  contribute  to  the  support  of  those 
functions  from  which  they  obtain  benefit.  The  activities  performed  during 
the  transition  period  were  reported  in  RADC-TR-83-132,  Data  and  Analysis 
Center  for  Software:  An  IAC  in  Transition. 

IITRI  was  awarded  the  current  36-month  contract  in  1982  to  continue 
operating  the  DACS  as  a  full-scale  IAC.  The  contract  was  amended  to  extend 
the  performance  period  to  40  months.  This  report  addresses  the  activities  of 
the  DACS  from  9  December  1982  to  8  April  1986,  the  current  contract  period. 

1.2  OBJECTIVES  OF  THE  DACS 

The  DACS  was  established  to  serve  as  a  focal  point  for  software 
development  and  experience  data;  for  the  analysis,  synthesis  and 
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dissemination  of  this  data;  and  for  scientific  and  technical  information 
(STINFO)  concerning  the  field  of  software  engineering.  As  implemented,  the 
DACS  provides  a  centralized  authoritative  source  for  current,  readily  usable 
data  and  information  concerning  software  technology.  The  objectives  of  the 
DACS  are  to: 

o  Encourage  the  exchange  of  software  technology  information  among  DOD 
and  civil  government  agencies,  government  contractors,  the  private 
sector  and  academia 

o  Support  software  technology  research  by  providing  a  centralized 

source  of  software  life  cycle  data 

o  Increase  the  productivity  of  software  producers  and  the  quality  of 
the  resultant  computer  software  by  improving  the  transfer  of  software 
engineering  technology 

o  Assist  in  diffusing  new  technology  throughout  the  U.S.  industrial 
base,  thereby  expanding  its  capability  and  competitive  posture 

o  Provide  scientific  and  technical  information  analysis  services  to 

DOD,  civil  agencies,  government  contractors,  and  the  private  sector 
in  areas  relating  to  software  technology  needs,  developments,  and 
trends 

o  Minimize  duplication  of  software  technology  research 


Specific  objectives  for  improving  the  DACS  during  this  reporting  period 
included: 


o  Expanding  the  role  of  the  DACS  in  the  software  engineering 

community 

o  Increasing  the  quality  of  Scientific  and  Technical  Information 

(STINFO)  acquisition 

o  Improving  life  cycle  data  acquisition  and  analysis  capabilities 

o  Increasing  the  automation  of  routine  DACS  procedures 

o  Expanding  and  promoting  special  studies  capabilities 


1.3  REPORT  CONTENTS 


Activities  of  the  DACS  described  in  this  report  were  performed  with  the 
above  objectives  In  mind.  This  report  contains  11  sections  which  summarize 
the  DACS  activities  during  the  contract  period  extending  from  9  December 
1982  to  8  April  1986.  A  brief  description  of  the  topics  covered  in  each 
section  follows. 


Section  1.0 
Section  2.0 

Section  3.0 

Section  4.0 

Section  5.0 


Background  and  objectives  of  the  center. 

Summary  and  technical  progress  and  activities  involved  In  the 
operation  of  the  center. 

Descriptions  of  the  data  acquisition  program  and  the  DACS 
Software  Life  Cycle  Empirical  Database. 

Description  of  the  Scientific  and  Technical  Information 
Database. 

Discussion  of  the  DACS  data  analysis  program  and  related 
efforts. 


Section 

Section 


6.0 


7.0 


8.0 


Summary  of  the  current  awareness  program.  Including 
newsletters,  bulletins  and  technical  presentations. 

Discussion  of  various  DACS  products  and  services.  Including 
data  subsets,  data  compendlums,  state-of-the-art  reviews, 
bibliographic  and  tool  searches,  and  consulting  services. 


Presentation  of  new  DACS  products  and  services,  along  with  a 
discussion  of  improvements  of  products  and  services. 

Discussion  of  promotional  advertising  to  new  users  by  use  of 
free  publicity  and  presentations  at  conferences  and  symposia. 

Synopses  of  the  fourteen  DACS  special  efforts  performed  during 
this  contract  period. 


Section 

Section  9.0 
Section  10.0 

Section  11.0  Observations  and  recommendations. 
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TASK  1  -  OPERATION  AND  MAINTENANCE  OF  CENTER 


2.1  SUMMARY  OF  GOALS,  TECHNICAL  PROGRESS  AND  ACTIVITIES 

2.1.1  Goals  Set  for  the  DACS 


The  activities  of  the  DACS  during  this  period  were  oriented  toward: 

o  Disseminating  state-of-the-art  information  on  software  technology  of 
general  interest  to  the  software  engineering  community 

o  Conducting  a  user  awareness  program  through  publication  of 
newsletters  and  bulletins,  presentations  at  professional  seminars  and 
at  the  sites  of  potential  clients,  and  active  participation  in 
professional  and  technical  organizations 

o  Preparing  and  distributing  products  and  services  designed  to  meet  the 
information  needs  of  the  DACS  user  community 

o  Preparing  a  compendium  of  software  engineering  standards  to  include 
military,  civil  government  and  voluntary  standards 

o  Maintaining  and  expanding  the  Software  Life  Cycle  Empirical  Database 
(SLED),  containing  data  which  is  descriptive  of  the  development  and 
maintenance  process  of  a  variety  of  software  projects 

o  Maintaining  and  expanding  the  STINFO  database  to  provide  up-to-date 
research  materials  from  which  the  products  and  services  of  the  DACS 
are  to  be  developed  and  to  have  available  the  most  recent  information 
to  supply  to  users 

o  Expanding  the  Software  Tool  Information  (STI)  database,  offering 

custom  tool  searches  to  DACS  users 

o  Expanding  the  scope  of  data  analysis  activities  and  disseminating  the 
results  of  analysis  activities  to  the  software  community  in  a  readily 
usable  form 

o  Continuing  and  expanding  the  promotional  program  so  that  the  maximum 

number  of  potential  users  of  DACS  products  and  services  are  made 

aware  of  their  existence 

o  Continuing  solicitation  of  user  feedback  on  the  products  and  services 

offered  by  the  DACS  to  ensure  that  they  will  continue  to  provide  the 

greatest  possible  usefulness  to  DACS  users 

o  Tracking  and  tabulating  of  user  interactions  with  the  DACS  so  that 
the  optimum  mix  of  products  and  services  may  be  provided  to  the 
persons  and  organizations  who  support  the  DACS  through  the  purchase 
of  its  products  and  services 


2.1.2  Summary  of  Activities  Accomplished 


The  following  paragraphs  briefly  summarize  the  major  accomplishments  of 
the  DACS  during  this  contract  period. 

The  Software  Life  Cycle  Empirical  Database  (SLED)  software  was  enhanced 
to  Include  automatic  generation  of  tapes  and  listings  for  three  additional 
datasets.  Using  the  parameterized  software,  the  Independent  Verification  and 
Validation  (IV&V)  Dataset,  the  NASA/SEL  Dataset  and  the  Composite 
Productivity  Dataset  can  be  produced  by  clerical  personnel.  Additional 
productivity  data  has  been  added  to  the  DACS  Productivity  Dataset,  and  the 
DACS  obtained  three  new  datasets  for  the  SLED. 

The  review  of  several  technical  reports  on  data  collection  and  a  study  of 
the  SLED  documentation  led  to  the  production  of  the  DACS  Data  Collection 
Package.  This  package  Includes  forms  for  the  collection  of  software 
engineering  measurement  data. 

A  new  data  analysis  plan  was  developed  based  on  the  plan  produced  under 
the  previous  contract,  and  selections  were  made  from  the  suggested  data 
analysis  tasks  described  In  the  plan.  The  Baselines  Analysis  Task  has  been  a 
continuing  project  during  this  contract  period,  and  four  baseline  progress 
reports  were  produced.  The  reports  entitled  "A  Comparative  Study  of 
Parametric  and  Non-Parametrlc  Statistical  Procedures  in  the  Analysis  of 
Software  Engineering  Data,"  and  "A  Simulation  Methodology  for  Software 
Engineering  Analysis"  were  also  produced. 

The  on-line  bibliographic  database  was  expanded  to  over  6,000  documents. 
Hew  software  was  developed  to  automate  and  streamline  input  processes,  and 
the  custom  searches  were  enhanced  in  both  appearance  and  information  content. 
The  data  definition  for  the  software  tool  information  database  was  changed  to 
reflect  new  data  Items,  and  existing  data  was  restructured  to  coincide  with 
the  new  data  definition.  Additionally,  the  applications  software  for  the 
database  was  enhanced.  Information  on  224  new  tools  was  added  to  the 
database. 


The  OACS  produced  and  distributed  13  newsletters  and  26  bulletins  as 
part  of  the  current  awareness  program.  In  addition,  the  DACS  staff  presented 
papers  at  conferences  and  gave  presentations  at  the  sites  of  potential 
clients.  The  OACS  also  responded  to  publicity  questionnaires  for  free 
advertisement  of  Its  products  and  services.  Finally,  several  new 
promotional  fliers  and  brochures,  which  outline  many  of  the  DACS  products  and 
services,  were  produced  and  distributed. 

User  fees  during  this  contract  period  averaged  $2,708  per  month.  The 
total  number  of  technical  inquiries  processed  was  2,452,  which  significantly 
increased  the  monthly  technical  Inquiry  average  when  compared  to  the  last 
reporting  period. 

Fourteen  special  study  efforts  were  negotiated  and  Implemented  at  the 
DACS  during  this  contract  period.  Each  of  these  projects  is  discussed  in 
Section  10.0. 


3.0  TASK  2  -  ACQUISITION  OF  SOFTWARE  EXPERIENCE  DATA 

3.1  INTRODUCTION 

There  is  a  real  need  to  collect  productivity  and  failure  data  on  the 
development,  operation  and  maintenance  of  software  to  support  research  In  the 
software  field.  Data  is  needed  which  will  allow  researchers  to  Isolate  the 
factors  that  contribute  significantly  to  the  costs,  reliability  and  quality 
of  the  software,  to  measure  achieved  reliability,  to  predict  development  and 
maintenance  costs,  and  to  track  the  progress  of  a  software  development 
project.  This  section  contains  descriptions  of  the  DACS  data  acquisition 
program  and  the  resultant  DACS  Software  Life  Cycle  Empirical  Database  (SLED). 

3.2  DATA  ACQUISITION  APPROACH 

The  means  used  to  maintain  the  data  acquisition  program  consisted  of: 

o  Identifying  data  sources  and  acquiring  relevant  data 

o  Establishing  procedures  for  automatic  submission  of  data 

o  Establishing  procedures  for  processing  and  evaluating,  and 
for  database  entry 

o  Maintaining  the  computer  database 

These  procedures  were  documented  at  length  in  RADC-TR-81-385  (CAR082)  and 
will  not  be  reproduced  here;  the  reader  Is  referred  to  this  document  for  full 
details. 

3.3  SOURCES  IDENTIFIED 

The  DACS  has  made  numerous  inquiries  during  this  contract  period,  with 
the  intention  of  ascertaining  the  availability  of  software  data.  The 
Inquiries  Included  reviewing  data  sources  (primarily  Commerce  Business  Daily 
(CBO)  announcements),  contacting  potential  data  sources,  and  Initiating 
follow-up  activities  associated  with  these  data  sources.  The  acquisition 
activities,  along  with  meetings  conducted  to  discover  additional  data 
sources,  resulted  in  significant  new  datasets  for  the  SLED: 


..... -  - --  -  -  •  - •  • '  - 


o  The  DACS  Productivity  Dataset  contains  project  level  data  on  over  500 
projects 

o  The  Reliability  Dataset  contains  data  on  over  2,800  failures  from  16 
projects 

o  The  NASE/SEL  Dataset  contains  very  detailed  data  on  over  30  projects 
conducted  by  NASA's  Goddard  Space  Flight  Center 

o  The  Verification  and  Validation  (V&V)  Dataset  contains  data  on  almost 
1,000  anomaly  reports  from  five  software  projects 

o  The  Architecture  Research  Facility  (ARF)  Error  Dataset  contains  data 

on  117  errors  from  a  system  with  253  modules;  the  dataset  also 

contains  module  descriptive  data 

o  The  Baseline  Software  Dataset  (BSDS)  contains  problem  report  and 

module  data  on  six  large  software  projects 

o  The  PAVE  Phased  Array  Warning  Systems  (PAWS)  Operations  and 

Maintenance  (O&M)  Dataset  consists  of  data  collected  by  the  Program 
Support  Library  on  the  PAVE  PAWS  system 

o  The  NASA/Ames  Dataset  contains  data  on  over  3,700  software  problem 
reports  from  about  35  projects 

o  The  DACS  Composite  Productivity  Oataset  contains  data  from  the  DACS 
Productivity  Dataset,  the  NASA/SEL  Dataset,  and  the  V&V  Dataset;  the 
composite  dataset  explored  the  prospects  of  unifying  all  DACS 

datasets 


3.3.1  Automatic  Submission  of  Data 


Automatic  generation  of  data  for  the  PAVE  PAWS  and  NASA/SEL  Datasets  is 
continuing.  The  DACS  receives  updated  data  on  tape  from  the  PAVE  PAWS 
programming  agency  once  a  month,  and  NASA/SEL  sends  updated  data  on  tape 
approximately  once  a  year. 

3.4  ACCOMPLISHMENTS  OF  THE  DACS  SOFTWARE  LIFE  CYCLE  EMPIRICAL  DATABASE 
(SLED) 


DACS  personnel  expanded  the  SLED  by  114  projects,  69  of  which  were  added 
to  the  DACS  Productivity  Dataset  as  a  result  of  a  literature  search.  The 


remaining  45  projects  were  generated  by  aggregating  NASA/SEL  data  at  the 
system  level  and  were  also  added  to  the  DACS  Productivity  Dataset. 

The  SLED  was  Implemented  on  the  RADC  HIS  870  computer  system,  using 
Honeywell's  Management  Data  Query  System  (MDQS)  to  facilitate  the  management, 
manipulation,  preparation  and  analysis  of  the  data.  Software  was  written  to 
allow  clerical  personnel  to  produce  tapes  and  hardcopies  of  the  SLED 
datasets.  This  software  greatly  reduces  the  time  needed  to  process  orders  for 
these  datasets. 

DACS  personnel  also  corrected  various  anomalies  in  several  of  the 
datasets  and/or  the  documentation.  Finally,  the  DACS  continued  its  program 
of  automatically  collecting  updated  data  from  both  NASA/SEL  and  the  PAVE  PAWS 
programming  agency. 

3.5  CURRENT  STATUS  OF  THE  DACS  SLED 

The  DACS  SLED  currently  consists  of  nine  sets  of  data  distinguishable  by 
data  source,  data  collection  and  acquisition  methodology,  life  cycle  phase 
represented,  and  data  parameters  present.  Because  each  set  of  data  was  the 
result  of  a  data  collection  effort  that  pursued  individually  specific 
objectives,  the  resulting  datasets  differ  with  regard  to: 

o  The  time  period  represented  by  projects  in  a  dataset 

o  The  portion  of  the  software  life  cycle  represented  by  the  data 

o  The  aspects  of  the  software  development  and/or  maintenance  processes 
measured  by  the  data  collection  activity 

o  The  quality  of  the  data  as  reflected  in  the  verification  and  validation 
procedures  used  in  data  collection 

o  The  subsequent  analyses  supported  by  the  data 

The  nine  data  sources  associated  with  the  SLED  are  listed  below.  Complete 
descriptions  of  the  datasets  can  be  found  in  the  references  provided. 

(1)  The  DACS  Productivity  Dataset  -  data  collected  from  various 
government  and  private  industry  sources  and  complied  by  Richard 
Nelson  of  RAOC  (NELS78) 


(2)  The  Reliability  Dataset  -  data  collected  at  Bell  Laboratories, 
Whlppany,  N.J.  and  complied  by  John  Musa  (MUSA79) 

(3)  The  NASA/SEL  Life  Cycle  Dataset  -  data  collected  and  contributed  by 
the  Software  Engineering  Laboratory  (SEL)  at  NASA  Goddard  Space 
Flight  Center  (BASI79) 

(4)  The  Verification  &  Validation  (V&V)  Dataset  -  data  collected  under 
several  Independent  V&V  contracts  then  summarized  and  delivered  to 
the  DACS  by  Log Icon  Incorporated  (RADA81) 

(5)  The  ARF  Error  Dataset  -  data  collected  and  analyzed  on  the 
development  of  the  Architecture  Research  Facility  (ARF)  at  the  Naval 
Research  Laboratories  (NRL)  by  David  Weiss  (EL0V79) 


(6)  The  Baseline  Software  Dataset 
(DUVA79 I , DUVA79 II) 


'BSPS)  -  data  collected  on 


(7)  The  Operations  and  Maintenance  (0&M1  Dataset  -  data  collected  on  the 
operations  and  maintenance  of  the  PAVE  PAWS  Phased  Array  Warning 
System  (IITR82) 

(8)  The  DACS  Composite  Productivity  Dataset  -  data  combined  from  the 
DACS  Productivity,  NASA/SEL  and  IV&V  datasets  as  part  of  feasibility 
study  into  streamlining  the  SLED  (TURN82) 

(9)  The  NASA/Ames  Dataset  -  data  on  over  3,700  software  problem  reports 
from  approximately  35  projects 

When  analyzing  data  of  this  nature.  It  Is  important  to  consider  that  the 
datasets  were  generated  at  different  points  in  time.  Figure  3.5-1 
illustrates  the  periods  of  time  represented  by  data  In  each  of  the  datasets. 
Each  of  the  datasets  contains  data  from  various  software  life  cycle  phases  as 
depicted  In  Figure  3.5-2.  Of  these  nine  datasets,  six  are  available  in  a 
standard  format.  The  remaining  three,  the  ARF,  BSDS,  and  O&M  datasets, 
because  of  their  extensive  nature,  have  not  been  processed  into  a  form  which 
Is  readily  usable,  which  limits  distribution  to  customized  versions  of  these 
datasets.  Each  of  the  nine  sets  of  data  is  discussed  in  detail  In  The  DACS 
Data  Compendium  (TURN82). 
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OACS  PRODUCTIVITY  DATASET 


4.0  TASK  3  -  SCIENTIFIC  AND  TECHNICAL  INFORMATION  (STINFO) 


4.1  INTRODUCTION 

Scientific  and  Technical  Information  (STINFO)  consists  of  documented 
Information  concerning  the  state-of-the-art  and  technology  aspects  of  the 
computer  software  field.  STINFO  usually  Includes  technical  reports,  trade 
journal  publications,  proceedings  of  conferences  and  symposia,  theses,  texts, 
and  product  descriptions  and  specifications.  Table  4.4-1  shows  the  sources 
from  which  the  software  engineering  documents  are  Identified.  Also  Included 
as  STINFO  are  descriptions  of  ongoing  software  technology  research  for  which 
reports  may  not  yet  have  been  produced.  These  two  types  of  STINFO  serve  as 
Input  to  three  Information  databases  maintained  by  the  DACS:  the  Software 
Engineering  Bibliographic  (SEB)  Database,  the  Software  Engineering  Research 
Projects  (SERP)  Database,  and  the  Software  Tool  Information  (STI)  Database. 

4.1.1  The  Software  Engineering  Bibliographic  (SEB)  Database 

The  SEB  Database  was  established  to  provide  a  readily  accessible  source 
of  comprehensive  Information  of  the  state  of  the  art  In  software  engineering, 
as  well  as  a  means  of  channeling  that  Information  to  those  people  in  the 
software  engineering  community  who  can  make  use  of  It  In  their  day-to-day 
activities  of  developing,  maintaining,  and  managing  software.  The 
bibliographic  collection  Is  composed  of  texts,  technical  reports,  theses, 
journal  articles,  proceedings  and  other  documents  relating  to  software 
engineering,  reliability,  costs  and  quality  factors,  maintainability,  and 
other  topics  deemed  appropriate.  The  collection  Is  computer-accessible  and 
retrieval  of  document  Information  can  be  made  on  any  part  of  the  document 
citations  or  on  assigned  keywords. 

New  software  for  the  SEB  Database  was  written  to  automate  the  Input 
process  and  the  error  checking  process.  Also,  the  existing  software  was 
modified  to  accommodate  the  addition  of  a  new  field  which  determines  the 
document  processing  status.  The  addition  of  this  field  allows  the  entry  of 
Incomplete  summarized  documents  Into  the  database  and  eliminates  a  separate 
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TABLE  4.1-1. 


SOFTWARE  ENGINEERING  DOCUMENTS  ARE  IDENTIFIED  FROM 
MANY  SOURCES 


Source  of  Document 

Type  of  Document 

Identification  Method 

U.S.  Government 

Nat' 1  Technical  In¬ 
formation  Service 
(NTIS) 

Unclassified  gov't  reports 
prepared  by  gov't  agencies 
and  their  contractors  or 
grantees 

Regular  review  of  publicizing 
instruments  NTIS  Techni¬ 
cal  Notes 

Defense  Technical  In¬ 
formation  Center 
(OTIC) 

Classified  or  limited  dis¬ 
tribution  reports  on  A&D 
done  by  DoO  or  Its  con¬ 
tractors 

OTIC  Technical  Abstracts 
Bulletin 

General  Accounting 

Office 

Investigative  reports  on 
activities  of  government 
agencies 

DACS  Is  on  distribution  for 
reports  pertinent  to  soft¬ 
ware  technology 

Naval  Publications  and 
Forms  Center 

OoO  standards,  regulations 
and  specifications 

Catalog 

RAOC 

Technical  Reports 

Distribution  list 

Professional  Society 

Papers  Conference  proceed¬ 
ings,  Journals,  tutorials 

Membership  In  Society 

Attendance  at  selected  con¬ 
ferences,  workshops, 
symposia 

Review  catalogs  of  publica¬ 
tions 

Publisher  of  Periodi¬ 
cals  directly  rele¬ 
vant  to  software 
engineering 

Journal  articles,  papers 

Subscription 

Publisher  of  Texts 

Texts 

Catalog  or  announcement 
or  book  review  In  profes¬ 
sional  journal 

Colleges  and  Unlver- 

Theses  and  Dissertations 

Periodic  review  of 

sltles 

■ 1  inTTSTl  4 FTTw.'  Tl-iV  ■ 

by  University  Microfilms 

Information  Retrlval 
Services 

Papers,  conference  pro¬ 
ceedings,  journals, 
tutorials 

Subscription 

In-process  database.  Streamlining  the  SEB  Database  resulted  In  a  significant 
reduction  of  clerical  time  required  for  document  processing.  During  this 
contract  period  2,725  documents  were  added  to  the  SEB  Database.  Figure  4.1-1 
displays  the  composition  of  the  SEB  Database. 


4.1.2  The  Software  Enqlneerinq  Research  Projects  (SERP1  Database 


The  OACS  maintains  the  SERP  to  provide  a  computer-accessible  source  of 
Information  about  recent  and  ongoing  research  in  the  field  of  software 
engineering.  Projects  covered  in  this  database  are  those  involving  software 
technology  research,  such  as  the  development  or  evaluation  of  programming 
languages,  models  or  software  tools,  and  research  related  to  software 
engineering  methodologies,  such  as  modern  programming  practices.  The 
database  was  developed  during  the  pilot  period  and  was  refined  during  the 
transition  period  of  the  DACS  operation. 


4.1.3  The  Software  Tool  Information  (STI1  Database 


The  National  Bureau  of  Standards  (NBS)  developed  a  software  tools 
database  which  contained  information  on  250  software  tools.  NBS  could  no 
longer  support  this  database  and  was  seeking  a  new  facility  which  could 
support  its  tools  database  and  make  it  available  to  the  software  engineering 
community.  Recognizing  the  need  for  a  central  source  of  software  development 
tool  Information,  the  DACS  acquired  NBS's  Software  Development  Tools 
Database,  which  formed  the  foundation  for  the  STI  Database. 


The  STI  Database  was  updated  during  this  contract  period.  Requests  for 
current  Information  on  the  tools  were  sent  to  all  sources  that  we.  *  used  in 
the  Initial  database  development.  Responses  received  were  used  +o  delete 
tools  that  were  no  longer  available  and  to  modify,  correct  and  add 
information  on  other  tools. 
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To  facilitate  the  collection  of  Information  on  new  software  development 
tools,  the  OACS  developed  Software  Tool  Description  Forms.  These  forms  are 
used  In  conjunction  with  an  Information  acquisition  program  Implemented 
specifically  for  Identifying  and  acquiring  Information  on  new  software 
development  tools.  OACS  personnel  produced  the  Software  Life  Cycle  Tools 
Directory,  which  Is  distributed  as  a  DACS  product. 

The  activities  associated  with  updating  the  STI  Database  and  acquiring 
new  tool  Information  for  the  database  resulted  In  approximately  224  new  tools 
being  added,  53  tools  being  deleted  and  24  tools  being  updated.  Table  4.1-2 
displays  a  breakdown  of  the  types  of  tools  In  the  STI  Database. 

TABLE  4.1-2.  STI  TOOL  TYPES. 


Number  In  This 


Tool  Class  Class 

Software  Management  Control  &  Maintenance  132 

Software  Modeling  &  Simulation  15 

Requirements  and/or  Design  Specification  57 

&  Analysis 

Source  Program  Analysis  &  Testing  117 

Program  Construction  &  Generation  50 

Software  Support  System/Programming  Environments  50 

Total  Number  of  Tools  421 


5.0  TASK  4  -  DATA  ANALYSIS  PROGRAM 


5.1  INTRODUCTION 

The  DACS  contributes  to  software  technology  research  through  data 
analysis,  as  well  as  through  special  studies,  state-of-the-art  reviews,  and 
technology  assessments.  The  first  data  analysis  program  plan  was  initiated 
in  1981.  During  the  current  reporting  period,  a  second  data  analysis  program 
plan  was  produced  by  J.  Romeu  of  the  DACS  (R0ME83),  which  builds  on  and 
further  develops  the  initial  plan. 

These  data  analysis  efforts  provide  input  for  long-term  research  and 
development  objectives,  which  include: 

o  Providing  a  better  understanding  of  the  processes  involved  in 

producing,  managing  and  maintaining  software 

o  Identifying  those  factors  which  influence  the  cost  quality, 

reliability  and  complexity  of  software 

o  'Developing  and  improving  methods  and  tools  for  use  in  producing  and 
maintaining  software 

o  Applying  or  developing  new  tools  for  the  data  analysis  effort 

o  Developing  techniques  for  estimating  costs,  resource  requirements  and 
schedules  of  future  software  products  and  methods  for  predicting  the 
reliability  of  those  projects 

o  Developing  techniques  to  assist  managers  in  planning,  measuring,  and 
tracking  of  development  and  maintenance  of  software  projects 

This  section  provides  an  overview  of  the  data  analysis  program  plan  and 
discusses  the  research  and  development  efforts  which  resulted  from  this  task. 

5.2  OVERVIEW  OF  THE  DACS  DATA  ANALYSIS  PROGRAM  PLAN 

The  purpose  of  the  Data  Analysis  Program  Plan  for  the  DACS  (R0ME83)  was 
to  outline  a  set  of  analysis  tasks  which  could  be  performed  at  the  DACS.  The 
SLED  served  as  primary  input  to  all  the  analyses  outlined  in  the  plan. 
Figure  5.2-1  shows  the  analysis  process  that  Is  carried  out  for  each 
investigation. 
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Figure  5.2*1.  Analysis  Process 
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Experience  with  previous  data  analysis  tasks  permitted  a  better 
assessment  concerning  the  availability  and  reliability  of  the  data  In  the 
SLED  Database  and  In  turn  made  It  possible  to  define  areas  In  which  more 
relevant  or  useful  results  could  be  obtained.  New  topics  and  areas  to  which 
future  analysis  could  be  directed  were  also  derived  from  previous  analysis 
experience. 

The  plan  delineated  four  major  topic  areas: 

(1)  Environment  and  Technology  Studies 

(2)  Cost  Study 

(3)  Baseline  Analysis  Activities 

(4)  Technical  Monographs 

The  Environment  and  Technology  Studies  consisted  of  both  research  and 
development  tasks.  One  effort  was  chosen  from  each  of  the  task  areas  In  the 
Environment  and  Technology  Studies. 

5.3  DATA  ANALYSIS  EFFORTS 

During  this  contract  period  the  DACS  performed  several  data  analysis 
projects  which  established  procedures,  methods,  and  routines  that  support  the 
Investigation  of  software  metrics,  provide  data  summary,  and  assess  and 
compare  software  engineering  tools  and  methods. 

DACS  personnel  developed  two  dataset  evaluation  metrics,  INTEGRATION  and 
COVERAGE,  which  are  used  to  determine  the  adequacy  of  the  data  In  one  of  the 
SLED  datasets,  to  do  comparisons  across  a  variety  of  projects,  and  to 
determine  If  the  database  contained  data  elements,  as  required  by  the  various 
software  reliability  models  and  other  analytical  techniques. 

The  DACS  Data  Compendium,  which  describes  the  SLED  datasets,  the  data 
Items  contained  In  them,  and  high  level  summaries  of  the  datasets,  was 
produced  and  was  updated  to  reflect  changes  made  to  the  SLED  during  this 
contract  period. 
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An  evaluation  of  equivalent  parametric  and  nonparametrlc  statistical 
tests  was  performed  with  respect  to  their  appropriateness  for  analyzing 
software  engineering  data. 

For  much  of  this  contract  period,  ongoing  data  analysis  performed  by  DACS 
personnel  produced  software  baselines  from  the  SLED  MASA/SEL  dataset.  These 
baselines  show  how: 

o  Development  resource  use  varies  with  the  size  of  the  module 

o  Subjective  estimates  of  complexity  are  reflected  In  more  objective 

variables  of  size  and  resources  expended 

o  More  precise  specifications  lead  to  less  complex  tasks  and  to  the  use 
of  fewer  resources 

o  Estimates  change  during  the  course  of  the  life  cycle 


ipj  iFjyw  b>ww  pj  pj  wumHMuiiw 


WWWW^^WWV 


twjtwmw 


6.0  TASK  S  -  CURRENT  AWARENESS  PROGRAM 


6.1  INTRODUCTION 


The  current  awareness  program  was  maintained  and  enhanced  throughout 
this  report  period.  The  program  has  two  purposes:  (1)  to  keep  the  DACS  user 
community  Informed  of  the  latest  and  most  significant  developments  In 
software  technology  and  software  engineering,  and  (2)  to  Inform  Its  current 
user  community  and  potential  users  of  the  products  and  services  offered  by 
the  DACS,  as  well  as  the  benefits  to  be  realized  through  use  of  the  DACS. 
The  DACS  Implements  the  current  awareness  program  In  the  following  ways: 

o  Publication  of  the  DACS  Newsletter 


o  Publication  of  the  DACS  Bulletin 
o  Presentations  at  conferences  and  symposia 

o  Establishment  of  contacts  throughout  the  software  engineering 
community  through  active  participation  In  professional 
organizations 

o  Placement  of  press  releases,  announcements,  and  paid  advertisements 
concerning  DACS  products  and  professional  activities  in 
professional  journals,  newspapers,  and  magazines  circulated  to  the 
software  engineering  community 

o  Publication  and  dissemination  of  Informational  materials  designed 
and  developed  by  the  DACS  staff 


Activities  relating  to  the  dissemination  of  information  are  discussed  in 
this  section;  the  promotional  aspects  of  the  current  awareness  program  are 
discussed  under  Task  8,  Promotional  Advertising. 


6.2  DACS  NEWSLETTER 


The  newsletter  is  the  primary  means  for  the  dissemination  of  current 
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Information  to  the  DACS  user  community.  Each  issue  contains  synopses  and 
critiques  of  significant,  newly  acquired  reports  or  articles;  summaries  of 
new  R&O  programs;  listings  of  future  conferences  and  symposia;  summaries  of 
significant  technological  breakthroughs  and  new  technological  applications; 
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and  highlights  of  other  outstanding  developments  of  Interest  to  DACS  users. 
The  DACS  Newsletter  Is  also  used  as  a  vehicle  to  announce  new  DACS  products 
and  programs  to  its  user  community.  One  DACS  product  is  featured  in  each 
Issue,  with  capsule  summaries  of  other  products  featured  as  space  permits. 

The  DACS  Newsletter  is  distributed  free  of  charge  to  government  and 
nongovernment  personnel  who  are  interested  in  the  disciplines  served  by  DACS. 
The  DACS  Newsletter  is  mailed  to  the  more  than  2,200  individuals  and  is  also 
distributed  at  conferences  attended  by  DACS  personnel.  Figure  6.2-1  shows 
the  front  page  of  the  December  1985  Issue  of  the  DACS  Newsletter. 

6.3  DACS  BULLETINS 

The  first  DACS  Bulletin  was  published  in  April  1979,  and  since  then  50 
Issues  have  been  published,  26  issues  during  this  contract  period.  Figure 
6.3-1  Illustrates  the  front  page  of  the  January  1986  issue  of  the  DACS 
Bulletin. 

The  bulletin  has  been  regularly  distributed  on  a  limited  basis  to 
RADC/CO  personnel  and  selected  individuals.  The  DACS  Bulletin  usually  treats 
a  topic  of  high  Interest  In  greater  depth  than  space  allows  in  the 
newsletter.  Certain  issues  of  particular  Interest  to  our  general  user 
community  have  been  publicized  In  the  newsletter  and  were  distributed  on 
request  for  a  nominal  charge.  Table  6.3-1  provides  a  list  of  the  DACS 
Bulletins  produced  during  this  contract  period. 

6.4  TECHNICAL  PRESENTATIONS 

Fourteen  technical  presentations  were  made  by  DACS  personnel  during  this 
reporting  period.  Technical  presentations  are  an  effective  way  to  inform  the 
technical  community  of  the  concerns  and  activities  of  the  DACS  and  to  attract 
new  users.  They  also  serve  to  Identify  areas  of  concern  in  the  DACS  user 
community,  new  sources  of  data  for  the  data  acquisition  program,  and  Inputs 
for  the  newsletters  and  bulletins.  In  addition  to  technical  presentations  at 
client  sites  and  technical  symposia,  presentations  were  made  at  the  DACS  to 
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EDITOR'S  NOTES 

Thu  edition  of  th*  newsletter  is  for  the  mott 
part  devoted  to  informing  our  readers  about  the 
availability  of  certain  documents  they  may  be 
interested  in.  A  new  feature  includes  a 
description  of  one  of  the  tools  included  m  the 
0AC5  Software  Tool  information  (STI)  Database 
(See  September  85  newsletter),  interest  m  such  a 
feature  was  strongly  expressed  in  our  DACS  Users 
Survey. 

GLOSSARY  Of  SOFTWARE 

— 

According  to  the  IEEE  Standard  Glossary  of 
Software  Enomeenno  Terminology,  software 
quality  is  the  totality  of  features  and 
characteristics  of  a  software  product  that  bear  on 
its  ability  to  satisfy  given  needs.  Considerable 
research  hat  been  conducted  in  the  area  of 
software  quality  measures  and  metres,  in  spite  of 
this  research,  no  universally  accepted  set  of 
measures  for  software  quality  yet  exists.  In  fact, 
the  quality  factors  and  attributes  that  have  been 
proposed  <n  this  research  vary  m  both  taxonomy 
and  definition.  The  Data  •  Analysis  Center  for 
Software  (DACS)  hat  produced  a  draft  'Glossary  of 
Software  Quality  Terms*  to  provide  a  single 
reference  for  definitions  of  terminology  related  to 
software  quality. 

This  glossary  includes  terms  from  tome  of  the 
earliest  work  m  this  area  to  some  of  the  most 
recent  work.  These  terms  are  derived  from  efforts 
related  to  software  quality  sponsored  by  Rome  Air 
Development  Center  (RA0Q.  the  National  Bureau 
of  Standards  (NBS).  and  the  Air  force  Operational 
Test  A  Evaluation  Center  (AEOTEQ.  The  glossary 
includes  the  definitions  found  in  the  draft  military 
standard  for  software  quality  evaluation  (DOO- 
STD-2J68).  Belated  terms  from  software 
engineering  glossaries  developed  by  the  National 
Aeronautics  and  Space  Administration  Software 


Engineering  Laboratory  (NASA/SEl)  and  the 
institute  of  Electrical  and  Electronics  Engineers 
Computer  Society  (IEEE-CS)  have  also  been 
included. 

Copies  of  the  glossary  are  available  from  the 
DACS.  This  glossary  is  presented  as  a  draft  to 
encourage  others  to  contribute  additional 
references.  Any  comments  on  sources,  format,  or 
scope  are  welcomed. 

C0MPSAC8S 

The  IEEE  Computer  Society's  Ninth 
International  Computer  Software  4  Applications 
Conference  (COMPSACBS)  was  held  from  October 
9  to  It.  1985  in  Chicago.  The  proceedings  are 
available  from 

IEEE  Computer  Society 
Post  Office  Box  80452 
Worldway  Postal  Center 
Lot  Angeles.  CA  90080 


The  first  OOOflndustry  STARS  Program 
Conference  was  hosted  by  the  National  Security 
Industrial  Association  (NSIA)  Software  Committee 
from  Apnl  30  to  May  2.  1985  Sessions  were  held 
in  each  of  the  six  thrust  areas:  applications, 
business  practices.  human  resources, 
measurement,  methodology,  and  software 
engineering  environments.  See  the  June  1985 
DACS  Newsletter  for  more  details  about  the 
conference.  Proceedings  for  this  conference  are 
now  available  for  S45  from 


NSIA  Headquarters 
Suite  901 

1015  15th  Street.  N  W 
Washington.  DC  20005 
202-393-362 
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NASA  SSL'S  TDfTH 


SOmMK  EMINCEAINB  WORKSHOP 


The  Software  Engineering  laboratory  (SEL)  at  NASA's  Goddam  Space  night 
Canter  (6SFC)  In  Greenbelt,  Maryland  hosted  the  Tenth  Annual  Software 
Engineering  Workshop  on  fleraatiai  4,  1985. 

John  Quant,  Oeputy  Director  of  NASA/GSFC.  we  leaned  the  group  with  renarks 
on  the  growth  of  the  workshop  to  Its  cun  ant  size  and  predictions  of 
continued  growth  in  the  future.  Oue  to  an  Increased  focus  on  Ada*  by  the 
Workshop,  a  growing  interest  In  Space  Station  software  and  Its  developeent 
environment.  and  an  axp«id1ng  International  awareness  of  the  need  to  stay 
currant  with  Ada.  Quann  predicted  a  need  far  larger  facilities  and  longer 
duration  for  the  1986  Workshop. 

The  Workshop  was  structured  around  four  sessions,  each  with  its  own 
topic.  Each  session  included  an  introducer  and  two  to  four  presenters. 

Jerry  Page  of  Coaputer  Sciences  Corporation  introduced  the  first  session 
and  gave  an  overview  and  history  of  the  SEl.  Since-  Its  astaOUsnnent  in 
1976,  experiments  have  bean  conducted  on  over  SO  projects,  ranging  in  size 
from  2.000  to  160,000  source  lines  of  code.  It  was  against  this  backdrop 
that  papers  on  the  first  topic.  Research  In  the  SEL.  were  presented. 

Victor  Basin  of  the  University  of  Maryland  delivered  the  first  talk  of 
the  opening  session,  measuring  the  Software  Process  and  Product:  lessons 
learned  In  the  SEL.'  The  rationale  for  aeasuring  software  and  its  developeent 
process  ware  said  to  Include  the  developeent  of  a  corporate  eeeory,  the 
determination  of  strengths  and  weaknesses  of  current  practices  and 
procedures,  the  developeent  of  a  rationale  for  adopting  techniques,  the 
alien— if  of  the  lapect  of  techniques,  and  the  evaluation  of  the  quality  of 
the  procedure  and  the  product. 

A  5- step,  goal-driven  paradigm  was  presented  for  data  collection:  (1) 
establish  goals,  (2)  develop  specific  questions  and  subgoals,  (3)  determine 
entries,  (4)  design  data  collection  fores,  and  (5)  collect  and  analyze  data. 


*Ada  Is  a  registered  trademark  of  the  Department  of  Defense  (Ada  Joint 
Progrms  Office). 


TABU  13-1.  TWENTY-SIX  ISSUES  OF  THE  OACS  BULLETIN  WERE  PRODUCED 
BY  DACS  STAFF  MEMBERS 


«su« 

TITLE 

AUTHOR 

January  ltd 

Noses  on  tbs  7th  Annual  NASA/geddard  Software  Engine  tring  Workshop 

J.  Romeo 

febnmry  INI 

RAMS  ISM  Pram  a  Combined  Hardware  Iritwaa  System  Perspective 

J.  Rameau 

Apr!  IMS 

Software  Ingiwoirtwg  trogrm be.  A  Mport  of  tha  SoXwirt 

Cfovp 

S.  Gloss- Solar 

May  IMS 

Naur  Saaks  at  tka  OACS 

S.  Gloss- Soier 

July  IMS 

Ra  part  aw  The  SacaM  Soffto ata  Engineering  Standards  AppAcaWoni  workshop 

S.  Gloss- Solar 

IryMIMS 

mOTWm 

G.  Sroment 

October  IMS 

Cast  WaRaklty  Tt aria  Off »:  A  Talk  by  Or.  terry  Soaluw 

1.  Rameau 

Neeemktr  IMS 

Iriluara  Coat  Sarimarian 

R.  luomo 

January  IMa 

Criiaria  fur  Software  AedaMky  Modal  Comparisons:  A  trvuw 

J.  Romeu 

Palmary  IMA 

Softness  AaRaMRty  at  RAMS  ISM 

J.  Romeu 

Apr*  IMA 

The  Ada*  Information  Oaaringhouaa 

E.  Swede 

May  ISM 

i 

1 

i 

i 

N.  Sunderhaft 

My  ISM 

Tha  OACS  Software  Engineering  Data  CoMaedon  Package 

N.  Sunderhaft 

August  IMA 

fortran  IX 

r  Robbins 

October  1M4 

AIM  Software  Portability  Survey 

T.  Robbins 

MuMMMrtM* 

Tha  Hamilton  CoMaga  Seminar  Series  on  Arbftciai  into  Riga  nta 

N.  Sunderhaft 
G.Rroment 

January  ISM 

Tha  fifdt  Oanaradon  Projact  After  the  f lm  Three  Yean 

N.  Sunderhaft 

Pefcruery  IMS 

COMPS  AC  ISM  Prssentariow 

E.fedchak 

April  ISM 

rVWnn  AfWMHf  acnwifg  EirfMVIlUiy  IVUI  RSnOfl 

J.  Romeu 

May  ISM 

N.  Sunderhaft 

AuguaMMS 

eighth  Mkmourhrooa  Workshop  on  Software  Performance  E  valuation 

R.  Vienneau 

OcaakariSM 

Software  Ufa  Cydo  Took 

G.  Srement 

November  IMS 

ffloatary  of  Software  Quality  Terms 

T  Robbins 

January  ISM 

NASA  SCL's  Tenth  Annual  Software  Engineering  Work  shoo 

0  Preston 

february  ISM 

SCI  Hosts  Software  factory  forum 

R.  Sunderhaft 

April  ISM 

NSIA  and  National  Institute  on  Software  Quality  and  Productivity  Conferences 

R.  Vienneau 

persons  or  groups  visiting  Grlfflss  Air  Force  Base  for  meetings  and 
workshops.  A  sample  of  DACS  presentations  can  be  found  In  Table  6.4-1. 


6.5  SUMMARY 

This  vigorous  current  awareness  program  has  resulted  in  several  benefits 
to  the  DACS. 

o  DACS  personnel  have  established  a  network  of  contacts  throughout  the 
software  engineering  community  and  are  regularly  invited  to  make 
presentations  at  conferences  and  symposia 


o  DACS  personnel  have  developed  an  In-depth  knowledge  of  the  needs  and 
concerns  of  the  DACS  user  community;  this  knowledge  Is  needed  to 
prepare  and  publish  newsletters  and  bulletins  which  are  both 
Informative  and  useful 

o  DACS  personnel  have  developed  production  and  quality  control  procedures 
to  assure  that  newsletters  and  informational  materials  are  of  high 
quality  In  both  appearance  and  technical  content 

o  Through  participation  In  technical  symposia,  workshops,  and 
professional  activities,  DACS  personnel  have  acquired  a  broad  base  of 
Information  regarding  the  activities  of  Individuals  and  organizations 
In  the  field  of  software  engineering. 
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7.0  TASK  6  -  PRODUCTS  AND  SERVICES  PREPARATION  AND  DISTRIBUTION 


7.1  INTRODUCTION 

This  section  sunmarizes  the  results  of  the  tasks  to  produce  and 
distribute  the  DACS  products  and  services.  A  characterization  of  these 
products  and  services  is  presented,  along  with  summarized  quantitative 
information  on  requests  processed. 

7.2  DATA  SERVICES 

Computer  readable  and  hard  copy  subsets  of  the  data  contained  in  the  DACS 
SLED  are  produced  and  distributed  in  response  to  specific  requests  to  aid  in 
research  efforts  that  require  productivity,  cost,  complexity,  error  and 
change  data.  Typically,  these  datasets  are  used  to  validate  and  refine 
software  reliability,  maintainability,  and  estimation  models  and  to  aid  in 
additional  data  analysis  studies  that  require  empirical  data. 

When  a  dataset  is  requested,  the  description  of  which  data  items  and 
type,  the  sorted  order,  and  other  information  must  be  known;  if  the 
description  is  not  included  in  the  request,  DACS  personnel  contact  the  user 
directly.  During  the  pilot  period  a  history  of  requests  was  kept.  As 
patterns  of  use  developed,  standard  options  were  identified  to  ease  the 
processing  load  on  engineering  personnel.  Parameterized  HOL  procedures  were 
written,  allowing  clerical  personnel  to  produce  data  subsets  for  standard 
options.  Additional  options  have  been  identified,  and  parameterized 
procedures  for  additional  datasets  were  produced  during  this  contract  period. 
The  DACS  has  distributed  125  copies  of  SLED  datasets  in  hard  copy  report 
format  or  on  magnetic  tape. 

To  facilitate  distribution  of  these  datasets  and  their  subsets, 
descriptive  literature  on  SLED  datasets  is  provided  to  the  potential 
purchaser  along  with  an  order  form.  Upon  receipt  of  the  order,  the  tape  or 
hardcopy  listing  is  generated.  The  data  is  then  sent  along  with  a  data 
dictionary  describing  the  data  elements. 


a 


1 


i 


The  DACS  Data  Compendium  Is  a  summarization  of  the  data  contained  In  the 
SLED.  It  contains  descriptions  of  the  datasets  (described  In  Section  3.5  of 
this  report),  DACS  products  and  services  ordering  Information,  and  record 
formats  for  all  the  datasets  described. 

DACS  personnel  periodically  updated  information  in  the  compendium  to 
reflect  any  changes  made  to  the  descriptions  of  data  contained  in  the  SLED. 
The  third  and  fourth  editions  of  the  compendium  were  produced  during  this 
contract  period. 

7.3  STATE-OF-THE-ART  REVIEWS 

An  effective  approach  to  stimulating  user  interest  in  the  DACS  is  through 
the  design,  preparation  and  distribution  of  products  that  constitute  the 
authoritative  sources  of  information  needed  throughout  the  software 
engineering  community.  State-of-the-art  reviews  (SOARs)  are  intended  to 
consolidate  and  synthesize  Information  on  a  specific  high-interest  technology 
area  from  multiple  sources  into  a  single  document  containing  all  pertinent 
Information  In  a  condensed,  easy-to-assimllate  form.  SOARs  are  most  useful 
when  the  topic  areas  they  cover  have  received  a  substantial  amount  of 
attention  by  scattered  researchers  (indicating  a  broad  interest),  but  the 
research  objectives  and  results  have  not  been  previously  consolidated.  In 
fast  moving  technologies,  it  may  be  appropriate  to  update  a  SOAR 
periodically. 

Six  such  SOARs  were  produced  during  this  contract  period: 

(1)  The  DACS  Software  Engineering  Data  Collection  Package 

(2)  Software  Life  Cycle  Tools  Descriptions 

(3)  Software  Engineering  Standards  Analysis:  An  Application  to  the 
Development  of  a  High  Quality  System 

(4)  Application  of  Tools,  Technologies  and  Methodologies  for  Rule-Based 
Expert  Systems 


W 


(5)  A  Review  of  Software  Engineering  Environments 

(6)  Determination  of  Data  Collection  Needs  for  the  Ada  Program. 

7.4  SOFTWARE  ENGINEERING  BIBLIOGRAPHY 

Scientists  and  engineers  depend  upon  well-designed  bibliographies  for 
ready  access  to  previous  work  and  published  literature.  To  be  most  useful, 
entries  must  be  comprehensively  indexed  with  terms  that  are  pertinent  to  and 
In  common  use  within  the  community  being  served.  In  response  to  this  need, 
the  DACS  developed  the  DACS  Thesaurus,  a  standard  word  list  used  for  indexing 
documents  In  the  DACS  library  and  has  published  and  disseminated  The  DACS 
Annotated  Bibliography.  The  DACS  bibliography  set  currently  consists  of  an 
intltlal  volume  and  four  supplements,  three  of  which  were  produced  during 

this  contract  period.  The  total  number  of  citations  included  in  the  set  is 
5,513.  Bibliography  volumes  contain  a  section  having  a  citation  and  abstract 

for  each  document,  a  copy  of  the  thesaurus,  an  index  of  subjects,  an  index  of 

authors,  and  a  keyword- In-context  listing  of  the  document  titles. 
Instructions  for  using  the  bibliography  and  document  ordering  information  are 
also  Included.  The  set  of  published  bibliographies  is  available  to  DACS 

users. 

7.5  BIBLIOGRAPHIC  SERVICES 

Bibliographic  inquiries  to  the  DACS  are  received  in  many  forms:  by 
letter,  telephone  call,  and  visit,  or  by  use  of  the  bibliographic  request 
form  contained  in  The  DACS  Annotated  Bibliography.  The  information  requests 
have  ranged  from  very  specific  to  general  questions  on  software  engineering 
methodologies.  The  DACS  has  conducted  51  bibliographic  searches  during  this 
contract  period.  DACS  personnel  have  also  provided  consultation  services  on 
109  requests  for  blbl lographic  assistance. 

Bibliographic  inquiries  are  processed  on  a  timely  basis,  with  a  search 
strategy  generated  and  an  on-line  search  of  the  DACS  database  normally  made 
within  one  or  two  days  from  receipt  of  an  inquiry.  Inquiries  are  tracked  to 
ensure  that  response  time  does  not  exceed  ten  days;  responses  may  take  the 
form  of  a  letter  or  listing.  In  those  cases  where  the  user  needs  immediate 


response  by  telephone  or  telegraph,  a  copy  of  the  telegram  or  a  memorandum- 
for-record  Is  filed.  Abstracts  are  supplied  with  the  custom  bibliographies, 
which  are  reviewed  by  a  member  of  the  DACS  technical  staff  to  ensure  that  the 
material  Is  pertinent  to  the  subject.  The  DACS  has  developed  efficient 
search  procedures  and  has  refined  them  to  provide  more  focused  bibliographies 
In  response  to  specific  user  requests. 

7.6  CUSTOM  TOOL  SEARCHES 

During  this  contract  reporting  period,  the  DACS  continued  the  custom 
search  service  for  software  tools.  A  custom  search  involves  automated 
searches  of  the  DACS  STI  Database. 

The  information  provided  on  each  software  tool  includes:  tool  title 
and/or  acronym,  classification,  features/functions,  stage/date  of 
development,  applicability,  implementation  language,  portability,  size 
hardware,  restrictions,  availability,  an  abstract  or  summary,  documentation, 
contact,  and  developer.  The  DACS  performed  50  custom  tool  searches  during 
this  reporting  period. 

7.7  TECHNICAL  INQUIRIES 

Technical  inquiries  to  the  DACS  are  received  by  mail,  telephone  call,  or 
in  person  and  are  processed  on  a  daily  basis.  Information  requests  range 
from  general  inquiries  to  very  specific  questions. 

Technical  inquiries  are  answered  in  different  ways,  depending  on  the 
nature  of  the  inquiry  and  the  complexity  of  the  reply.  Responses  include: 

o  Custom  bibliographic  search  on  the  subject  area  of  interest 

o  Summary  information  supplied  based  on  a  preliminary  analysis  of  the 
subject  literature 

o  Production  of  a  subset  of  a  DACS  database 

o  Distribution  of  relevant  DACS  literature 

o  Referral  to  other  sources  of  information 


Figure  7.7-1  shows  the  types  of  inquiries  received  by  the  DACS  during 
each  month  of  this  contract  period. 

Certain  inquiries  of  a  very  specific  technical  nature  may  not  be 
answerable  in  terms  of  a  dataset,  a  bibliographic  search,  or  a  published  DACS 
product,  but  may  be  answered  relatively  quickly  by  a  DACS  specialist.  For 
such  inquiries,  DACS  provides  engineering  services  in  the  form  of  technical 
guidance,  with  additional  DACS  products  included,  as  appropriate.  A  total  of 
2,452  technical  inquiries  were  processed  during  this  contract  period.  The 
DACS  responds  to  technical  inquiries  within  ten  working  days. 

7.8  TECHNOLOGY  ASSESSMENTS 

Technology  assessments  are  special  studies  that  generally  exceed  the 
scope  of  a  technical  inquiry  response,  but  are  usually  shorter  than  a  state- 
of-the-art  review.  They  are  directed  toward  evaluating  and  synthesizing  the 
latest  available  information  resulting  from  recent  research  and  development 
findings  or  are  comparative  assessments  based  on  technical  characteristics. 
For  example,  a  technology  assessment  may  involve  the  extraction  and 
compilation  of  specialized  data  from  several  independent  works  to  evaluate 
their  differences;  it  may  be  a  synopsis  of  pertinent  findings  from  available 
documentation;  or  it  may  compare  different  approaches  to  a  life  cycle 
process,  using  documented  specifications  and  results. 

The  titles  of  the  six  technology  assessments  produced  during  this 
contract  period  are: 

(1)  A  Compendium  of  Software  Engineering  Standards 

(2)  A  Simulation  Approach  for  the  Analysis  and  Forecast  of  Software 
Productivity 

(3)  An  Approach  to  the  Evaluation  of  Software  for  Embedded  Systems 

(4)  Programming  Languages  for  Artificial  Intelligence  Systems 

(5)  An  Overview  of  DOD-STD-2167 

(6)  Calibration  of  the  C0C0M0  Cost  Calibration  Model. 


8.0  TASK  7  -  PREPARING,  DISTRIBUTING,  REVIEWING  AND  IMPROVING  PRODUCTS  AND 
SERVICES 

8.1  PREPARING  AND  DISTRIBUTING  PRODUCTS  AND  SERVICES 


A  vital  component  of  a  responsive  information  analysis  center  is  its 
ability  to  provide  products  that  reflect  the  needs  of  its  users.  The 
following  new  products  and  services  were  Introduced  to  DACS  users  during  this 
contract  period: 

o  JLC  Software  Development  Specifications  -  Versions  II,  III  -  a  draft 
of  four  standards  and  supporting  Data  Item  Descriptions  aimed  at 
providing  a  disciplined  software  development  process  for  DOD  software 
acquisition 

o  DACS  Data  Compendium  -  a  description  of  software  experience  data 
available  from  the  DACS;  includes  types  of  data,  number  of  records  of 
each  type,  and  record  formats 

o  KERNAL  Ada  Program  Support  Environment  (KAPSE)  Interface  Team  Public 
Report  -  Version  II,  III  -  a  report  published  by  the  KAPSE  Interface 
team  (KIT),  whose  objectives  are  to  evaluate  and  establish  KAPSE 
standards 

0  Software  Technology  for  Adaptable.  Reliable  Systems  (STARS)  -  revised 
plan  for  the  STARS  program;  ten  reports  packaged  as  one  volume 

o  DIANA  Reference  Manual  -  introduction  and  reference  manual  for  a 

Descriptive  Intermediate  Attributed  Notation  for  Ada  (DIANA) 

o  Ada  Bibliography  -  citations  and  abstracts  for  over  350  documents 

pertaining  to  the  history,  development,  progress,  and  uses  of  the  Ada 
language 

o  Catalog  of  Resources  for  Education  in  Ada  and  Software  Engineering 
(CREASE)  -  Vol.  I,  II  -  catalog  of  courses,  seminars,  and  training 
programs  on  Ada  and  software  engineering 

o  Draft  Specification  of  the  Common  APSE  Interface  Set  (CAIS)  - 
specification  for  a  set  of  Ada  packages  which  together  form  a  CAIS 
for  Ada  Programming  Support  Environments  (APSEs) 


DACS  Software  Engineering  Data  Collection  Package  -  produced  by  the 
DACS  to  help  organize  the  data  items  necessary  to  support  analysis 
activities  Into  a  classification  scheme  and  to  promote  standardized 
collection  of  software  engineering  data 
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o  DACS  Annotated  Bibliography  -  Vol.  Ill,  IV  -  a  collection  of 

citations  and  abstracts  of  literature  on  software  engineering  and 
software  technology 

o  Software  Development  Standards  (SDS)  Documentation  Set  -  a  set  of 

four  standards  and  the  supporting  data  Item  descriptions 

o  The  Software  Engineering  Life  Cycle  Tools  Directory  -  a  directory  of 
over  400  software  tools  contained  in  the  Software  Tool  Information 
(STI)  database  maintained  by  the  DACS 


8.2  IMPROVEMENT  OF  DACS  PRODUCTS  AND  SERVICES 


The  products  and  services  of  the  DACS  are  continually  reviewed  and 
assessed  with  the  objective  of  enhancing  both  the  DACS  and  its  products  and 
services.  Concerns  of  the  enhancement  efforts  extend  to  all  facets  of  the 
OACS  operations,  including  Improving  the  technical  quality  of  DACS  products. 
Increasing  the  scope  of  data  coverage,  improving  analysis  routines,  improving 
data  handling  capability,  operating  more  efficiently,  and  increasing  services 
to  the  user  community.  During  this  contract  period  significant  improvements 
have  been  made  in  several  areas.  These  improvements  include  the  following: 

o  Increasing  the  scope  of  data  coverage  by  the  acquisition  of: 

NASA-Ames  Data 
-  AIRMICS  Data 

Jet  Propulsion  Laboratory  Data 

o  Improving  analysis  routines 

o  Improving  software  pertaining  to  DACS  STINFO  Databases 

o  Increasing  efficiency  in  record-keeping  achieved  by  the  maintenance 
of  the  on-line  User  Profile  Database 

o  Streamlining  the  process  for  entry  of  bibliographic  information  into 
the  bibliographic  database 

o  Reviewing  technical  content  of  final  drafts  by  IITRI  personnel  who 
have  expertise  in  software  technology  but  who  are  not  members  of  the 
DACS  core  group 

o  Reviewing  of  the  final  draft  of  all  planned  publications  by  a 
technical  editor,  with  respect  to  style,  clarity  of  writing, 
correctness  of  grammar  and  spelling 
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8.3  CONTINUING  ASSESSMENT  OF  USER  NEEDS 


The  DACS  has  used  two  methods  to  determine  the  needs  of  Its  users:  (1) 
surveys  to  Identify  user  requirements,  and  (2)  historical  records  of  user 
requests.  These  two  activities  have  been  pursued  to  ensure  that  DACS 
products  and  services  meet  the  needs  of  the  user  community. 

User  surveys  conducted  by  the  DACS  have  Included  questions  which 
characterize  user  organizations,  job  descriptions,  and  areas  of  concern  or 
interest  In  the  field  of  software  engineering.  Free  response  questions 
elicit  suggestions  for  new  products  and  services.  Figure  8.3-1  Illustrates 
the  distribution  by  organization  and  occupation  of  DACS  users  who  responded 
to  a  June  1984  survey. 

The  most  recent  user  survey  was  conducted  in  December  1985,  and  responses 
to  this  survey  are  still  being  collected  and  analyzed.  An  earlier  survey 
conducted  In  June  1984  elicited  over  600  responses  from  the  2,000 
questionnaires  distributed,  a  very  high  level  of  response  for  this  type  of 
survey.  Figure  8.3-2  shows  the  forms  used  for  the  two  surveys.  User 
responses  were  carefully  reviewed  and  tabulated  to  determine  areas  of 
Interest  and  concern.  Areas  of  Interest  identified  by  the  June  1984  User 
Survey  are  shown  In  Fig.  8.3-3.  In  general,  this  survey  showed  that 
Industry  Is  seeking  technology,  tools,  and  Information  with  which  to  solve 
the  very  problem  the  DACS  was  created  to  address:  how  to  produce  high- 
quality,  reliable  software  at  a  reasonable  cost. 

Figure  8.3-4  Illustrates  the  Service  Request  Record,  which  is  used  to 
track  and  record  services  performed  by  the  DACS  in  terms  of  the  user 
(requestor),  the  type  of  service  provided,  and  the  resources  expended.  This 
information  Is  transferred  to  the  User  Profile  Database  (UPD)  to  accumulate 
the  types  of  services  provided  to  each  user.  Periodic  tabulations  of  total 
user  Interactions  with  the  DACS  are  generated  from  the  database  and  are 
correlated  with  the  results  of  user  surveys  to  evaluate  DACS  responsiveness 
to  user  needs. 
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Figure  8.3*2. 


Two  User  Surveys  (June  1984  and  December  1985)  Were  Used  to 
Identify  the  Software  Engineering  Areas  of  Greatest  Interest  to 


DACS  Customers  and  to  Develop  New  Products  and  Services 
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Figure  8.3-4.  The  Service  Request  Record  Captures  the  History  of  a  Completed 
User  Request  for  DACS  Products  and  Services 
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9.0  TASK  8  -  PROMOTIONAL  ADVERTISING 


9.1  PROMOTION  TO  NEW  USERS 


Since  it  is  unlikely  that  all  potential  users  of  the  DACS  have  been 
identified  or  are  on  the  mailing  list  for  its  newsletter,  ongoing  efforts  are 
necessary  to  reach  potential  users.  Potential  users  are  defined  as 
organizations  or  individuals  known  to  participate  in  research  and  development 
or  to  use  materials  within  the  scope  of  the  DACS,  who  have  a  high  potential 
for  benefit  from  the  use  of  DACS  products  and  services. 

Promotion  to  persons  and  organizations  in  the  potential  user  category 
was  accomplished  using  the  following  techniques: 

o  Placement  of  press  releases  and  announcements  in  journals, 
magazines  and  newspapers 

o  Presentations  at  conferences  and  symposia 

o  Procurement  of  mailing  lists  to  which  brochures,  catalogs  or  other 
promotional  materials  could  be  sent 

9.1.1  Promotion  by  Use  of  Free  Publicit 


There  are  several  newspapers  and  magazines  circulated  free  to  software 
engineers  that  will  print  press  releases  and  new  product  information  at  no 
charge  to  the  DACS.  In  addition  to  sending  announcements,  fliers  and 
brochures  to  several  of  these  journals,  editors  of  other  journals  and 
magazines  requested  Information  from  the  DACS  on  its  products  and  services. 

Specific  Instances  of  free  publicity  Include: 


The  March  1984  and  June  1984  Language  Control  Facility  Ada- Jovial 
Newsletter  announced  several  new  DACS  products 

The  DACS  Newsletter  received  free  publicity  in  the  January  1984 
issue  of  Software  Maintenance  News  (published  by  the  DPMA  Special 
Interest  Group  on  Software  Maintenance) 

The  June  1983  Issue  of  the  Communications  of  the  ACM  announced 
Software  Technology  for  Adaptable,  Reliable  Systems  (STARS) 
documents  soon  to  be  available  from  the  DACS 


o  Custom  searches  on  the  DACS  Software  Tool  Information  (STI) 
database  were  referred  to  In  the  May  1983  Issue  of  Computer 


In  addition  to  the  free  publicity  mentioned  above,  the  DACS  produced  and 
distributed  free  promotional  fliers  with  information  packets  to  reach 
potential  users.  Figures  9.1-1  and  9.1-2  show  the  promotional  fliers  and 
brochures  that  were  produced  during  this  contract  period.  The  DACS 
Newsletter  continued  to  be  used  as  a  regular  promotional  device.  Each 
newsletter  contains  book  reviews,  state-of-the-art  surveys,  conference 
announcements  along  with  featured  DACS  products  (an  article  used  to  introduce 
new  DACS  products),  and  capsule  summaries  of  selected  products  and  services. 
The  graph  In  Figure  9.1-3  shows  user  receipts  by  month.  There  is  a 
noticeable  increase  in  receipts  when  quarterly  newsletters  or  brochures  are 
distributed,  indicating  that  these  distributions  are  effective  means  of 
promoting  DACS  products  and  services. 

9.1.2  Presentations  at  Conferences  and  Svmoosia 


This  topic  Is  more  fully  discussed  under  Task  5,  Current  Awareness.  It 
Is  only  to  be  noted  here  that  these  presentations  provided  an  excellent 
opportunity  for  one-on-one  discussions,  as  well  as  personal  communication 
with  groups  of  potential  users  concerning  the  DACS  and  its  products  and 
services.  The  success  of  this  promotional  method  indicates  that  it  is 
worthwhile  to  continue  to  use  presentations  at  conferences  and  symposia  as  a 
vehicle  for  promoting  the  DACS. 
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10.0  TASK  9  -  SPECIAL  STUDIES  AND  PROJECTS 


There  are  many  problems  related  to  software  technology  that  can  be  solved 
through  the  full  service  capabilities  provided  by  the  DACS.  Many  problems 
are  sizable,  requiring  a  substantial  expenditure  of  engineering  resources  to 
accomplish.  The  DACS  can  make  a  significant  contribution  to  increasing  the 
productivity  of  software  engineers  and  researchers  and  at  the  same  time  can 
make  efficient  use  of  accumulated  information  and  resources. 

The  DACS  can  best  serve  its  user  community  by  identifying  those  areas  in 
government  and  industry  for  which  there  is  a  need  for  the  technological 
expertise  and  other  resources  of  DACS,  and  by  performing  special  studies 
tailored  to  the  individual  needs  of  particular  organizations.  Fourteen 
special  studies  were  initiated  during  this  contract  period  and  are  discussed 
in  this  section. 

10.1  ESTABLISH  A  MEASUREMENT  FRAMEWORK  FOR  THE  STARS  PROJECT 

This  DACS  effort  was  conducted  under  contract  to  the  Rome  Air  Development 
Center  (RADC)  for  the  DOD  STARS  Joint  Program  Office  (SJPO).  The  SJPO  tasked 
the  DACS  to  support  the  STARS  Software  Measurement  Task  by  developing  a 
coherent  and  complete  methodology  which  includes  the  collection,  storage, 
maintenance  and  dissemination  of  software  data.  Specific  objectives  of  this 
project  were  to  develop  a  set  of  Data  Item  Descriptions  (DIDs)  for  the 
collection  of  software  engineering  data  on  DOD  software  acquisition  and 
support  programs,  and  to  establish  a  data  collection  coordination  function 
for  the  STARS  Measurement  Project,  within  the  present  structure  of  the  DACS. 

After  successfully  meeting  these  objectives,  the  DACS  continued  to 
support  the  STARS  Software  Measurement  Task.  Results  of  these  support 
efforts  Included  the  "STARS  Measurement  Survey  Summary,"  which  presents  the 
results  of  a  survey  conducted  by  the  DACS  to  determine  the  state  of  practice 
of  software  measurement  in  the  defense  software  development  community;  the 
"Software  Measurement  Repository,"  a  report  providing  a  high-level  overview 
for  the  establishment  of  a  central  database  of  software  data;  and  the  "STARS 


Measurement  Annotated  Bibliography,"  a  hard  copy  bibliography  containing 
citations  and  abstracts  for  measurement-related  literature. 

10.2  SOFTWARE  QUALITY  ANALYSIS 

The  Software  Quality  Analysis  Project  was  performed  for  the  U.S.  Army 
Development  and  Readiness  Command  (DARCOM).  Under  Phase  I  of  this  project 
the  OACS  performed  a  metric  measurement  of  DARCOM' s  Long  Range  Budget 
Planning  (LRBP)  tool  to  assess  the  software  quality  of  the  tool. 
Enhancements  and  modifications  were  made  to  upgrade  the  LRBP  tool  as  a  result 
of  the  metrics  measurement,  and  software  data  was  collected  during  the  update 
process. 

Phase  II  Involved  Installing  the  updated  software  tool  at  the  designated 
sites,  training  personnel  In  its  use,  and  collecting  data  In  the  operations 
and  maintenance  (0&M)  environment.  Much  time  and  effort  was  saved  by 
automating  a  previously  tedious,  error-prone,  and  time-consuming  procedure. 

During  Phase  III  of  this  effort  metrics  data  on  maintainability  and 
flexibility,  and  on  additional  0&M  data  was  collected  on  a  module-by -module 
basis  on  the  tool  resulting  from  Phase  II.  All  data  collected  during  phase 
III  was  Included  In  the  DACS  SLED  to  facilitate  further  evaluation  and  future 
analysis. 

10.3  DETERMINATION  OF  DATA  COLLECTION  NEEDS  FOR  THE  ADA  ENVIRONMENT 

This  special  study  was  completed  under  contract  to  the  Rome  Air 
Development  Center  for  the  Ada  Joint  Program  Office  (AJPO).  The  AJPO 
required  a  set  of  data  collection  procedures  to  be  used  for  collecting  data 
on  projects  using  the  Ada  language.  A  methodology  was  developed  for 
collecting  data  In  a  consistent  manner,  allowing  the  collected  data  to 
support  data  analyses  that  would  produce  software  development  assessments  for 
the  Ada  language. 

As  a  result  of  this  study,  the  DACS  produced  a  report  entitled 
"Determination  of  Data  Collection  Needs  for  the  Ada  Program:  A  Methodology 


for  Identification  and  Data  Collection."  The  report  presents  a  generalized 
methodology  for  data  collection,  offers  insights  needed  to  apply  the 
methodology  effectively,  and  demonstrates  its  application  to  problems  of 
specific  interest  to  Ada  data  collectors. 


10.4  SYSTEM  ACQUISITION  GUIDELINES  FOR  THE  DESIGN  PHASE  OF  LAYAWAY 
COMPATIBLE  EPCS  (AMCCOM  -  LAMP-F) 


This  special  study  was  performed  for  the  U.S.  Army  Armament  Research 
Development  Command  (AMCCOM)  in  support  of  their  Layaway  and  Mobilization  for 
the  Future  (LAMP-F)  program.  The  objective  of  the  work  was  to  develop  a 
handbook  for  the  acquisition  manager  who  is  acquiring  systems  that  must 
satisfy  the  requisites  of  the  layaway  process. 


The  layaway  process  concerns  ammunition  plants  which  have  been  built 
using  electronic  process  control  computers  and  software.  The  plants  are 
"laid-away"  In  peacetime  and  remobilized  when  ammunition  is  needed.  This 
project  was  designed  to  ensure  that  the  software  which  controls  the 
ammunition  producing  processes  Is  of  adequate  quality  to  be  maintainable  in 
the  long-term  future.  A  large  amount  of  data  was  collected  and  was  included 
in  the  LAMP-F  handbook  as  a  case  study. 


10.5  SOFTWARE  QUALITY  PREDICTION  AND  ASSESSMENT 


This  special  study,  performed  under  contract  to  the  Software  Engineering 
Section,  Command  and  Control  Division  of  Rome  Air  Development  Center, 
developed  a  relationship  between  software  quality  prediction  and  assessment. 


This  effort  was  divided  Into  two  parts,  the  first  part  being  devoted  to 
the  development  of  a  methodology  for  measuring  the  cost  of  software  quality 
for  the  three  factors  of  reliability,  flexibility,  and  maintainability.  The 
second  part  was  directed  toward  the  characterization  of  the  three  quality 
factors  In  terms  of  measurable  properties  exhibited  by  software  during  the 
operations  and  maintenance  phase  of  the  software  life  cycle.  General 
Research  Corporation  provided  technical  support  on  this  effort. 
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10.6  SOFTWARE  QUALITY  BENCHMARK 


The  primary  objective  of  this  project  performed  for  the  Software 
Engineering  Section,  Rome  Air  Development  Center,  was  to  describe  a  procedure 
to  obtain  a  software  reliability  benchmark.  Tasks  for  this  study  Included 
defining  software  reliability  using  operational  measures,  describing  the  data 
needed  to  develop  and  validate  the  reliability  value,  and  developing  the 
procedure  for  collecting  the  data  and  performing  the  validation.  General 
Research  Corporation  provided  technical  support  for  this  effort. 

10.7  SOFTWARE  ENGINEERING  AND  ADA  LANGUAGE  STUDIES 
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The  objective  of  this  project  Is  to  support  the  Battlefield  Automation 
Office  of  Headquarters,  U.S.  Army  Materiel  Command  (AMC)  by  obtaining, 
analyzing,  and  evaluating  Information  on  activities  sponsored  by  AMC,  related 
to  software  engineering,  and  software  development  and  maintenance 
methodologies. 

OACS  personnel  Investigated  and  evaluated  engineering  methodologies  and 
tools  currently  used  at  Life  Cycle  Software  Engineering  (LCSE)  centers.  The 
Constructive  Cost  Model  (C0C0M0),  a  tool  used  to  predict  the  cost  of  software 
development  and  maintenance,  was  Implemented  at  LCSE  centers.  DACS  personnel 
then  developed  the  Software  Engineering  Cost  Model  (SECOMO),  a  version  of 
C0C0M0  which  was  specifically  calibrated  and  tailored  to  meet  the  needs  of 
the  LCSE  center's  applications. 

DACS  personnel  provided  training  In  the  use  of  the  model  at  the  various 
LCSE  centers  across  the  country.  DACS  involvement  in  this  effort  led  to  a 
DACS  report  entitled  "Calibration  of  the  C0C0M0  Cost  Estimation  Model,"  which 
explains  the  methodology  used  to  tailor  the  model  to  a  particular  LCSE 
application. 

10.8  SOFTWARE  QUALITY  ASSESSMENT  FOR  THE  ACIA  SYSTEM 


I 


Under  contract  to  the  Rome  Air  Development  Center  (RADC),  Signal 
Intelligence  (SIGINT)  Exploitation  Section,  Intelligence  and  Reconnaissance 
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Division,  the  DACS  completed  a  software  quality  assessment  for  the  ACIA 
system. 


The  objective  of  this  effort  was  to  perform  a  software  quality  analysis 
of  the  ACIA  software  system  for  the  purpose  of  achieving  a  predetermined 
quality  level.  Assessment  of  the  ACIA  system  was  made  using  automated 
procedures  to  perform  the  data  collection  and  analysis,  and  manual  review  of 
the  ACIA  source  code  and  documentation. 

The  desirable  quality  factors  of  a  software  product  vary  with  the  needs 
and  priorities  of  the  user.  The  DACS  determined  those  factors,  and 
identified  and  classified  those  characteristics  that  affect  the  determined 
software  quality  factors. 

As  a  result  of  this  study,  the  DACS  provided  RADC  with  recommendations 
for  achieving  the  predetermined  quality  level  in  the  ACIA  software. 

10.9  DOD-STD-SDS  DOCUMENTATION  SET 

The  objective  of  this  effort  was  to  support  the  Computer  Software 
Management  (CSM)  subgroup  of  the  Joint  Logistics  Commanders  Joint  Policy 
Coordinating  Group  on  Computer  Resource  Management  (JLC-JPCG-CRM)  by 
analyzing  a  public  review  of  the  Defense  System  Software  Development  Standard 
(SOS)  Documentation  Set  and  by  producing  a  revised  version. 

The  primary  tasks  performed  by  the  DACS  were  coordination,  development 
and  Implementation  of  a  current  awareness  program  to  Inform  the  DOD  and 
industry  of  the  status  and  plans  for  the  review  of  the  SDS  standards.  During 
the  process  of  transforming  the  SDS  Documentation  Set  Into  the  new  DOD-STD- 
2167,  support  consisted  of  special  bulletins.  After  DOD-STD-2167  was 
adopted,  the  DACS  continued  its  support  by  distributing  Information  on  the 
standard.  This  project  also  resulted  in  a  DACS  report,  "An  Overview  of  DOD- 
STD-2167,"  which  explains  how  the  new  standard  was  derived  and  how  It  Is 
used. 
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10.10  VALIDATION  OF  LOGICAL  STABILITY  METRICS 


The  objective  of  this  completed  study  was  to  collect  and  analyze  software 
measurements  for  validating  the  Logical  Stability  Metric.  This  effort  was 
performed  for  the  Software  Engineering  Section,  Command  and  Control  Division 
of  the  Rome  Air  Development  Center  (RADC). 

The  Logical  Stability  Metric  is  a  measure  of  the  resistance  of  a  program 
to  the  ripple  effect  (the  propagation  of  errors  after  a  program 
modification).  The  measure  is  influenced  by  a  program's  complexity. 
Potentially,  this  measure  can  be  used  to  indicate  the  time  and  effort  needed 
for  program  maintenance  and  to  identify  areas  within  a  program  that  are 
sensitive  to  modification. 

Software  programs  were  measured  by  the  logical  stability  metric,  using 
the  ripple  effect  analyzer  tool  (developed  by  Dr.  Yau  of  Northwestern 
University  for  RADC),  the  data  flow  analyzer,  and  the  program  complexity 
analyzer.  A  validation  plan  was  developed,  which  included  modifying  the 
programs,  collecting  data  to  quantify  the  modification  activities,  and 
analyzing  this  data  against  the  logical  stability  metric  for  the  same 
programs.  Experiments  were  conducted  for  the  tool  validation.  Several 
programs  were  subjected  to  three  test  methods;  these  tests  indicated  a 
correlation  between  the  logical  stability  measures  obtained  from  using  the 
software  tools  and  those  measures  obtained  from  the  experiments.  This  was  a 
joint  effort  with  Northwestern  University,  which  provided  much  of  the 
technical  support. 

10.11  SDS  FRAGMENTS/STRUCTURES 

As  part  of  the  STARS  program,  the  development  of  the  STARS  Software 
Engineering  Environment  (STARS-SEE)  is  intended  to  provide  support  across  the 
entire  software  life  cycle  and  to  increase  quality  and  productivity.  Under 
contract  to  Rome  Air  Development  Center,  the  DACS  provided  the  STARS  Joint 
Program  Office  with  a  background  analysis  for  the  STARS-SEE. 


This  effort  included  a  thorough  examination  of  the  software  life  cycle 
and  its  products  (data  items  and  data  fragments)  as  specified  by  DOD-STD- 
2167,  the  JLC  OIDs  and  the  STARS  Measurement  DIDs.  OACS  personnel  examined 
the  top-level  data  items  required  and  developed  consistent  definitions  for 
fragments  into  various  categories  and  a  mapping  of  fragments  onto  life  cycle 
activities. 

This  effort  continued  with  a  second  phase,  which  resulted  in  a  detailed 
examination  of  the  fragments  down  to  the  lowest  meaningful  level;  the 
classification  of  fragments  according  to  the  previously  developed  scheme, 
thus  creating  a  data  dictionary;  and  the  analysis  of  classified  fragments  in 
the  data  dictionary  to  determine  relationships  among  them,  such  as 
redundancy,  procedures,  and  derivablllty. 

10.12  SOFTWARE  ENGINEERING  FOR  STRATEGIC  DEFENSE  INITIATIVE 

The  purpose  of  this  project,  which  was  performed  under  contract  to  the 
Rome  Air  Development  Center,  Software  Engineering  Section,  Command  and 
Control  Division,  was  to  Identify  software  engineering  requirements  that  are 
responsive  to  the  timeliness  and  operational  performance  needs  dictated  by 
the  goals  of  the  Strategic  Defense  Initiative  (SDI).  In  particular,  the 
software  measurement  area  was  analyzed  in  detail  to  determine  how  it  can  best 
address  the  requirements  of  Battle  Management  Software. 

Project  tasks  included  precise  characterization  of  the  SDI  software 
environment,  comprehensive  analysis  of  the  state  of  the  art  In  software 
engineering  technology.  Identification  of  SDI-drlven  technology  voids  and 
deficiencies,  and  detailed  review  of  the  software  quality  measurement  area 
and  its  application  to  SDI  Battle  Management. 


10.13  DATA  COLLECTION  FOR  DECISIONS  AID  SOFTWARE 

To  support  the  STARS  Program  Office  In  the  Measurement  Thrust  Area,  Rome 
Air  Development  Center  tasked  the  DACS  to  provide  a  comprehensive  methodology 
for  the  collection  of  software  data. 

Software  data  collection  consists  of  the  measurement  or  monitoring  of  one 
or  more  aspects  of  the  development,  operations,  or  maintenance  of  a  software 
system.  The  collection  of  software  data  enabled  software  analysts  to  predict 
the  cost  of  software  development  and  maintenance,  to  verify  the  requirements 
originally  agreed  upon  between  the  customer  and  software  developer,  and  to 
determine  the  best  approaches  for  the  development  of  a  software  system  based 
on  the  trends  of  data  collected  from  past  and  present  software  development 
efforts. 

The  objective  of  this  effort  was  to  examine  an  existing  Battle  Staff 
Decision  Aids  software  project  for  the  long-range  goal  of  providing  a 
framework  that  can  be  used  to  effectively  evaluate  the  software  product  and 
process  throughout  all  phases  of  the  software  life  cycle. 

This  study  resulted  In  the  assembly  of  a  set  of  models  and  software 
metrics  that  measured  attainment  of  desired  characteristics. 

10.14  SYSTEM  INTERFACE  WITH  SIGINT  SUPPORT 

The  objectives  of  this  study  completed  for  the  Rome  Air  Development 
Center  were  to  analyze  the  software  quality  of  the  Dynamic  Ground  Target 
Simulator  (DGTS)  and  the  Advanced  Sensor  Exploitation  software  systems,  with 
respect  to  interoperability  and  reusability;  and  to  determine  the  feasibility 
of  Interfacing  these  software  systems  with  sensor  modeling  software. 

To  meet  these  objectives,  the  requirements  for  coupling  the  software 
systems  of  the  DGTS  and  the  SIGINT  SYSTEM  MODEL  (SSM)  were  determined.  Based 
on  Rome  Air  Developments  Center's  Software  Quality  Framework,  the  software 
quality  factors  of  Interest  were  Identified,  and  a  methodology  for  assessing 
these  factors  was  developed. 


m  jv  x'.vj  mi 


The  feasibility  of  coupling  the  software  systems  was  assessed  based  on 
the  results  of  the  software  quality  analysis.  This  study  resulted  In  the 
recommendation  of  a  specific  approach  for  coupling  the  systems  and  in  the 
demonstration  of  the  feasibility  of  that  approach. 

The  Interoperability  of  DGTS  and  the  SSM  modeling  systems  was 
accomplished  by  data  coupling  the  two  systems  and  by  reusing  existing 
software  programs.  The  techniques  used  did  not  affect  the  software  quality 
of  either  systems. 


11.0  OBSERVATIONS  AND  RECOMMENDATIONS 


11.1  OBSERVATIONS 

During  this  contract  period,  the  OACS  operated  as  a  full-scale  IAC, 
maintaining  user  Interest  as  reflected  In  the  average  monthly  receipts  of 
$2,708.  The  OACS  expanded  the  STINFO  and  SLED  databases  and  promoted  new 
products  and  services* related  to  these  databases  and  other  special  studies. 
The  need  for  Information  on  software  technology  continued  to  Increase,  which 
Implies  the  need  to  continually  expand  the  STINFO  database,  resulting  In  up- 
to-date  research  materials  from  which  the  products  and  services  of  the  OACS 
are  developed  and  the  availability  of  the  most  recent  Information  Is  supplied 
to  OACS  users.  An  area  which  relates  to  the  SLED  Database  is  software  data 
collection.  Encouraging  software  developers  to  collected  software  life  cycle 
empirical  data  Is  becoming  a  major  concern  of  the  DACS.  Several  special 
efforts  have  been  Initiated  to  develop  guidelines  for  data  collection,  and 
these  efforts  will  serve  as  a  data  acquisition  vehicle  for  the  expansion  of 
the  SLED  Database. 

Improvements  were  made  during  this  contract  period  which  included: 

o  Streamlining  data  entry  procedures  for  STINFO  databases  and 
standardizing  tape  procedures  for  SLED  datasets,  resulting  in  a 
significant  processing  time  reduction  of  data  for  both  of  these 
databases 

o  Streamlining  order-processing  procedures  to  reduce  duplication  of 
effort 

o  Developing  software  to  automate  repetitious  activities  of  the  DACS 
databases  and  office  procedures 


11.2  RECOMMENDATIONS 

The  following  recommendations  are  made  based  on  the  observations  made 
above  and  on  the  activities  described  during  this  period. 

o  Continue  to  operate  and  maintain  the  center  based  on  the  goals  set 
for  the  DACS,  and  actively  pursue  new  areas  of  software  engineering 
technology 
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Continue  to  expand  the  OACS  user  community  through  a  strong  and 
complete  current  awareness  program 

Conduct  special  related  efforts  In  order  to  expand  the  scope  of  the 
OACS 

Continue  to  expand  the  OACS  STINFO  and  SLED  databases  to  produce 
up-to-date  research  and  analysis  Information  from  which  the  DACS 
can  produce  products  and  services  that  supply  the  most  recent 
Information  to  DACS  users 

Encourage  software  developers  to  collect  software  life  cycle 
empirical  data,  and  continue  to  pursue  the  development  of 
guidelines  for  data  collection 

Continue  streamlining  clerical  and  administrative  procedures 
concerning  the  OACS  databases  and  office  procedures 

Identify  and  automate  repetitious  activities. 
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^  Rome  Air  Development  Center 

)  RAVC  plant  and  executes  research,  development,  test 

*  and  selected  acquisition  programs  in  support  oh 

1  Command,  Control,  Communications  and  Intelligence 
t  [Co i)  activities.  Technical  and  engineering 

>  support  within  areas  oh  competence  is  provided  to 

2  ESV  Program  Ohhicet  ( POs )  and  other  ESV  elements 

<  to  perhorm  acquisition  oh  C3 1  systems. 

S  The  areas  oh  technical  competence  include 

*  communications ,  command  and  control,  battle 

>  management,  inhcrmation  processing ,  surveillance 
i  sensors,  intelligence  data  collection  and  handling, 

>  soled  state  sciences,  electromagnetics ,  and 
)  propagation,  and  electronic,  maintainability , 

*  and  compatibility. 


